stockpyle A simple multi-layered storage and caching API
Download

stockpyle Ranking & Summary

stockpyle Tags

storage Python library caching multi-layered storage caching API multi-layered

stockpyle Description

A simple multi-layered storage and caching API stockpyle is a Python module that provides a simple way to set up a series of storage containers for the purposes of creating simple write-through cache storage.Usage:Simplest script that sets up a two-level cache with memcached and local process memory:from stockpyle import MultiStore, MemcacheStore, ProcessMemoryStore# instantiate the MultiStore as a write-through cachepm = ProcessMemoryStore()mc = MemcacheStore(servers=)store = MultiStore()# declare a class that is unique for each (bar,zap) combinationclass Foo(object): __stockpyle_keys__ = def __init__(self, bar, zap): self.bar = bar self.zap = zap# create and save a Foo to the MultiStoreobj = Foo(bar=444, zap=888)store.put(obj)# retrieve a Foo from the store, based on the (bar,zap) combination# this will hit the local memory cache first, and will avoid memcache# since the object is already cached thereretrieved_obj = store.get(Foo, {"bar": 444, "zap": 888})This example isn't that interesting, since we are using two caches. Let's do one that supports SQLAlchemy objects:from stockpyle.stores import MultiStore, SqlAlchemyStore, MemcacheStore, ProcessMemoryStorepm = ProcessMemoryStore()mc = MemcacheStore(servers=)sa = SqlAlchemyStore()store = MultiStore()# store it, this will write it through the cache and into the databasepersistent_obj = MySqlAlchemyBackedClass()store.put(persistent_obj)Note the ordering in the MultieStore declaration: the SQLAlchemyStore comes last, since it acts as the final persistence layer. Subsequent gets will attempt process memory, then attempt memcache, and finally check the database.Also, we can treat the process memory cache and the memcached differently by using different expirations. For example, you may want process memory to expire quickly, but memcached to last a little longer since you can actually keep it consistent across multiple machines. This example forces Foo objects to be expired more aggressively from the local memory than memcached:pm = ProcessMemoryStore()mc = MemcacheStore(servers=)sa = SqlAlchemyStore()store = MultiStore()# Foo objects will last 60 seconds in local memory, and 5 minutes in memcachepm.configure(classes=, lifetime=60)mc.configure(classes=, lifetime=5*60)Want to grab a bunch of objects? Use batch_get:obj1, obj2, obj3 = store.batch_get(Foo, )Want to store a bunch of objects? Use batch_put:obj1 = Foo(111, 777)obj2 = Foo(222, 888)obj3 = Foo(333, 999)store.batch_put()Deleting objects is easy (batch deletes coming soon):store.delete(obj1)You can use the Storable helper class to get class-level access to the storage APIs:from stockpyle import Storableclass Bar(Storable): __stockpyle_keys__ = def __init__(self, foo, bar): self.foo = foo self.bar = bar# bind to a storage objectstore = ProcessMemoryStore()Bar.bind(store)# save an objectobj = Bar(111, 777)obj.put()# get an objectobj = Bar.get({"foo", "bar"})# delete an objectobj.delete()# batched getobj1, obj2 = Bar.batched_get() Requirements: · Python

stockpyle Related Software