This software summary describes the BerkeleyDB Backend Storage Engine for DURUS, which offers a storage engine for DURUS, a persistence system specifically for Python.
The BerkeleyDB Backend Storage Engine for DURUS project is a storage engine that provides a persistence system for the Python programming language. It is an excellent alternative to the Durus standard FileStorage with many advantages. The startup time is negligible, and there is no need for an in-memory index. This means that your repository size is only limited by storage space, not RAM.
Furthermore, if you change existing objects, your storage size does not increase. If you delete objects, those objects are garbage collected in the background, slowly, without performance degradation. You can do a full fast collection if you need it, and while this collection is in progress, Durus still serves objects. From time to time, Durus can be unresponsive for 2 or 3 seconds while doing a checkpointing to be able to free database logging diskspace.
Another advantage is that the garbage collection does not increase storage size or RAM usage. Garbage collection deletes objects using nondurable transactions, which is very efficient. If the collection is aborted abruptly (program or machine crashes), the collection will start again from the beginning. If the garbage collection finishes without problems, that state is durable. Additionally, any object stored in the storage will commit a durable transaction, including all objects released in the background garbage collector along the way.
The disadvantage is that this backend uses reference counting to decide when an object is garbage and can be collected. If you have cycles in your data structures, you must break them before releasing the objects; otherwise, you will leak diskspace. Although leaking objects will grow the diskspace, there is no corruption or malfunction that will happen. It is possible that in a future release, we can collect cycles. Still, until then, try to avoid that pattern.
Another disadvantage is that sometimes this backend can become unresponsive for a couple of seconds. It is busy doing a checkpoint to recycle database logging space. The pause should be short, nevertheless. Don't use this storage backend over NFS unless you know what is going on.
Since we are using BerkeleyDB as the backend, you should be experienced with BerkeleyDB deployments. Beware when updating Python or BerkeleyDB because BerkeleyDB is known for breaking binary compatibility between versions. They always document the procedure to do a controlled upgrade. In this case, don't worry, but take note of the risk. To do a trustable backup, you should follow instructions in BerkeleyDB documentation.
You can use this product as a normal (local) filestorage or a server (remote) storage system, just like the usual Durus FileStorage. What's new in this release is compatibility with Durus 3.7.
Furthermore, if you change existing objects, your storage size does not increase. If you delete objects, those objects are garbage collected in the background, slowly, without performance degradation. You can do a full fast collection if you need it, and while this collection is in progress, Durus still serves objects. From time to time, Durus can be unresponsive for 2 or 3 seconds while doing a checkpointing to be able to free database logging diskspace.
Another advantage is that the garbage collection does not increase storage size or RAM usage. Garbage collection deletes objects using nondurable transactions, which is very efficient. If the collection is aborted abruptly (program or machine crashes), the collection will start again from the beginning. If the garbage collection finishes without problems, that state is durable. Additionally, any object stored in the storage will commit a durable transaction, including all objects released in the background garbage collector along the way.
The disadvantage is that this backend uses reference counting to decide when an object is garbage and can be collected. If you have cycles in your data structures, you must break them before releasing the objects; otherwise, you will leak diskspace. Although leaking objects will grow the diskspace, there is no corruption or malfunction that will happen. It is possible that in a future release, we can collect cycles. Still, until then, try to avoid that pattern.
Another disadvantage is that sometimes this backend can become unresponsive for a couple of seconds. It is busy doing a checkpoint to recycle database logging space. The pause should be short, nevertheless. Don't use this storage backend over NFS unless you know what is going on.
Since we are using BerkeleyDB as the backend, you should be experienced with BerkeleyDB deployments. Beware when updating Python or BerkeleyDB because BerkeleyDB is known for breaking binary compatibility between versions. They always document the procedure to do a controlled upgrade. In this case, don't worry, but take note of the risk. To do a trustable backup, you should follow instructions in BerkeleyDB documentation.
You can use this product as a normal (local) filestorage or a server (remote) storage system, just like the usual Durus FileStorage. What's new in this release is compatibility with Durus 3.7.
What's New
Version 20070503: N/A