[egenix-users] Calc size of Python objects

[M.-A. Lemburg]

Dirk Holtwick wrote:
> hi,
> 
> in mxTools there is a nice function called sizeof() to calculate the 
> amount of memory an object uses, but that's only true for simple objects 
> like strings and integers I believe. 

True. sizeof() only measures the size of the bare PyObject,
not including any possibly referenced memory. Since dictionaries,
classes, instances etc. use extra memory for keeping data,
the value returned by sizeof() is not correct.

Deep knowledge of how Python does its memory allocation is needed
to figure this out.

 > but I'm interested in the total
> amount a complex object uses. is this the right way to do it?
> 
> def calcsize(i, s=0):
>     s = sizeof(i)
>     if type(i) == type({}):
>         for k, v in i.items():
>             s += calcsize(k)
>             s += calcsize(v)

+ you have to add the memory for the dictionary table itself;
that's 2 PyObject pointers per slot. The table size depends
on the size of the dictionary.

>     elif type(i) == type([]):
>         for v in i:
>             s += calcsize(v)

For lists, sizeof() does not include the table of entries,
so you'll have to add one PyObject pointer per entry.
For tuples, sizeof() does include the table size.

>     return s
> 
> BTW, does someone have an idea how to get aware of the total memory 
> usage of a running python program within itself? the module "resource" 
> doesn't work on my machine.

Take a look at mx/ODBC/Misc/proc.py. That module is not open-source,
though, as it's part of mxODBC.

-- 
Marc-Andre Lemburg
CEO eGenix.com Software GmbH
_______________________________________________________________________
eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,...
Python Consulting:                               http://www.egenix.com/
Python Software:                    http://www.egenix.com/files/python/