[egenix-users] BeeDict memory usage

[M.-A. Lemburg]

Daniel Naber wrote:
> Hi,
> 
> I'm using BeeDict (egenix-mx-base-2.1.0b2) with the following code:
> 
> self.table = BeeDict(self.db_name, min_recordsize=0, readonly=0, \
> 	recover=0, autocommit=1, validate=0)
> 
> self.table is then used to save quite some (nested) data, i.e. lists and 
> dictionaries. This works well, but there's one problem: too much memory is 
> used when adding data. I had hoped that "on-disk" means something like: 
> save the data to disk if there's a flush() or commit() call so that the 
> data doesn't take up memory. I tried that, but it doesn't help (actually 
> commit()ing after adding a certain amount of data will leave the index 
> incomplete in the end).
> 
> Does anybody have an idea how to save memory when adding data?

BeeDicts keep an internal cache which could be the cause of the
memory footprint you are seeing. You can explicitly clear the
cache by calling .free_cache().

Perhaps it would be a good idea to have .flush() also free
the cache ?!

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/