From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Jan 13 11:13:17 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 13 11:13:17 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Jan 13 11:13:17 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 13 11:13:17 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Jan 13 11:13:17 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Jan 13 11:13:17 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 13 11:13:17 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Fri Jan 13 11:13:17 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 13 11:13:17 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Jan 13 15:13:50 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 13 15:13:50 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Jan 13 15:13:50 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 13 15:13:50 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Jan 13 15:13:50 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Jan 13 15:13:50 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 13 15:13:50 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Fri Jan 13 15:13:50 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 13 15:13:50 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Jan 13 15:21:12 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 13 15:21:12 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Jan 13 15:21:12 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 13 15:21:12 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Jan 13 15:21:12 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Jan 13 15:21:12 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 13 15:21:12 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Fri Jan 13 15:21:12 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 13 15:21:12 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Jan 13 16:00:44 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 13 16:00:44 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Jan 13 16:00:44 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 13 16:00:44 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Jan 13 16:00:44 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Jan 13 16:00:44 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 13 16:00:44 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Fri Jan 13 16:00:44 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 13 16:00:44 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Jan 13 20:01:30 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 13 20:01:30 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Jan 13 20:01:30 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 13 20:01:30 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Jan 13 20:01:30 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Jan 13 20:01:30 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 13 20:01:30 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Fri Jan 13 20:01:30 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 13 20:01:30 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Jan 14 08:02:12 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 14 08:02:12 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Jan 14 08:02:12 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 14 08:02:12 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Jan 14 08:02:12 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Jan 14 08:02:12 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 14 08:02:12 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sat Jan 14 08:02:12 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 14 08:02:12 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Jan 14 12:00:41 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 14 12:00:41 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Jan 14 12:00:41 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 14 12:00:42 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Jan 14 12:00:42 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Jan 14 12:00:42 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 14 12:00:42 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sat Jan 14 12:00:42 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 14 12:00:42 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Jan 14 16:00:41 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 14 16:00:41 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Jan 14 16:00:41 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 14 16:00:41 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Jan 14 16:00:41 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Jan 14 16:00:41 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 14 16:00:41 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sat Jan 14 16:00:41 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 14 16:00:41 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Jan 14 20:00:37 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 14 20:00:37 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Jan 14 20:00:37 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 14 20:00:37 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Jan 14 20:00:37 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Jan 14 20:00:37 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 14 20:00:37 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sat Jan 14 20:00:37 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 14 20:00:37 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Jan 15 08:02:02 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 15 08:02:02 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Jan 15 08:02:02 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 15 08:02:02 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Jan 15 08:02:02 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Jan 15 08:02:02 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 15 08:02:02 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sun Jan 15 08:02:02 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 15 08:02:02 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Jan 15 12:00:47 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 15 12:00:47 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Jan 15 12:00:47 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 15 12:00:47 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Jan 15 12:00:47 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Jan 15 12:00:47 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 15 12:00:48 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sun Jan 15 12:00:48 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 15 12:00:48 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Jan 15 16:00:41 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 15 16:00:41 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Jan 15 16:00:41 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 15 16:00:41 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Jan 15 16:00:41 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Jan 15 16:00:41 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 15 16:00:41 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sun Jan 15 16:00:41 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 15 16:00:41 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Jan 15 20:00:37 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 15 20:00:37 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Jan 15 20:00:37 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 15 20:00:37 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Jan 15 20:00:37 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Jan 15 20:00:37 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 15 20:00:37 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sun Jan 15 20:00:37 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 15 20:00:37 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Jan 16 08:02:25 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 16 08:02:25 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Jan 16 08:02:25 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 16 08:02:25 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Jan 16 08:02:25 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Jan 16 08:02:25 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 16 08:02:25 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Mon Jan 16 08:02:25 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 16 08:02:25 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Jan 16 12:00:45 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 16 12:00:45 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Jan 16 12:00:45 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 16 12:00:45 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Jan 16 12:00:45 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Jan 16 12:00:45 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 16 12:00:45 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Mon Jan 16 12:00:45 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 16 12:00:45 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Jan 16 16:00:42 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 16 16:00:43 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Jan 16 16:00:43 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 16 16:00:43 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Jan 16 16:00:44 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Jan 16 16:00:44 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 16 16:00:44 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Mon Jan 16 16:00:44 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 16 16:00:44 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Jan 16 20:00:36 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 16 20:00:36 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Jan 16 20:00:36 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 16 20:00:36 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Jan 16 20:00:36 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Jan 16 20:00:36 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 16 20:00:36 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Mon Jan 16 20:00:36 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 16 20:00:36 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Jan 17 08:02:26 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 17 08:02:26 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Jan 17 08:02:26 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 17 08:02:26 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Jan 17 08:02:26 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Jan 17 08:02:26 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 17 08:02:26 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Tue Jan 17 08:02:26 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 17 08:02:26 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Jan 17 12:00:54 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 17 12:00:54 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Jan 17 12:00:54 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 17 12:00:54 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Jan 17 12:00:54 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Jan 17 12:00:54 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 17 12:00:54 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Tue Jan 17 12:00:54 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 17 12:00:54 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Jan 17 16:00:45 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 17 16:00:46 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Jan 17 16:00:46 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 17 16:00:46 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Jan 17 16:00:46 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Jan 17 16:00:46 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 17 16:00:46 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Tue Jan 17 16:00:46 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 17 16:00:46 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Jan 17 20:00:46 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 17 20:00:47 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Jan 17 20:00:47 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 17 20:00:47 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Jan 17 20:00:47 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Jan 17 20:00:47 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 17 20:00:47 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Tue Jan 17 20:00:48 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 17 20:00:48 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Jan 18 08:04:09 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Jan 18 08:04:09 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Jan 18 08:04:09 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Jan 18 08:04:09 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Jan 18 08:04:09 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Jan 18 08:04:09 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Jan 18 08:04:09 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Wed Jan 18 08:04:09 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Jan 18 08:04:09 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Jan 18 12:00:58 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Jan 18 12:00:58 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Jan 18 12:00:58 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Jan 18 12:00:58 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Jan 18 12:00:58 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Jan 18 12:00:58 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Jan 18 12:00:58 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Wed Jan 18 12:00:58 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Jan 18 12:00:58 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Jan 18 16:01:20 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Jan 18 16:01:20 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Jan 18 16:01:20 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Jan 18 16:01:20 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Jan 18 16:01:20 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Jan 18 16:01:20 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Jan 18 16:01:20 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Wed Jan 18 16:01:20 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Jan 18 16:01:20 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Jan 18 20:00:47 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Jan 18 20:00:47 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Jan 18 20:00:47 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Jan 18 20:00:47 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Jan 18 20:00:47 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Jan 18 20:00:47 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Jan 18 20:00:47 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Wed Jan 18 20:00:47 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Jan 18 20:00:47 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Thu Jan 19 08:04:31 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Jan 19 08:04:31 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Thu Jan 19 08:04:31 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Jan 19 08:04:31 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Thu Jan 19 08:04:31 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Thu Jan 19 08:04:31 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Jan 19 08:04:31 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Thu Jan 19 08:04:31 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Jan 19 08:04:31 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Thu Jan 19 12:00:56 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Jan 19 12:00:56 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Thu Jan 19 12:00:56 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Jan 19 12:00:56 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Thu Jan 19 12:00:56 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Thu Jan 19 12:00:56 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Jan 19 12:00:56 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Thu Jan 19 12:00:56 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Jan 19 12:00:56 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Thu Jan 19 16:02:38 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Jan 19 16:02:39 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Thu Jan 19 16:02:39 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Jan 19 16:02:39 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Thu Jan 19 16:02:39 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Thu Jan 19 16:02:39 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Jan 19 16:02:39 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Thu Jan 19 16:02:39 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Jan 19 16:02:39 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Thu Jan 19 20:00:43 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Jan 19 20:00:43 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Thu Jan 19 20:00:43 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Jan 19 20:00:43 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Thu Jan 19 20:00:43 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Thu Jan 19 20:00:43 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Jan 19 20:00:43 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Thu Jan 19 20:00:43 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Jan 19 20:00:44 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Jan 20 08:06:21 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 20 08:06:21 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Jan 20 08:06:21 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 20 08:06:21 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Jan 20 08:06:21 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Jan 20 08:06:21 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 20 08:06:21 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Fri Jan 20 08:06:21 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 20 08:06:22 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Jan 20 12:01:52 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 20 12:01:52 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Jan 20 12:01:52 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 20 12:01:52 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Jan 20 12:01:52 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Jan 20 12:01:52 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 20 12:01:52 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Fri Jan 20 12:01:52 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 20 12:01:52 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Jan 20 16:00:47 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 20 16:00:47 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Jan 20 16:00:47 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 20 16:00:47 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Jan 20 16:00:47 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Jan 20 16:00:47 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 20 16:00:47 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Fri Jan 20 16:00:47 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 20 16:00:47 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Jan 20 20:00:43 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 20 20:00:43 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Jan 20 20:00:43 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 20 20:00:43 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Jan 20 20:00:43 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Jan 20 20:00:43 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 20 20:00:43 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Fri Jan 20 20:00:43 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 20 20:00:43 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Jan 21 08:03:45 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 21 08:03:45 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Jan 21 08:03:45 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 21 08:03:45 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Jan 21 08:03:45 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Jan 21 08:03:45 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 21 08:03:45 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sat Jan 21 08:03:45 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 21 08:03:45 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Jan 21 12:00:57 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 21 12:00:57 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Jan 21 12:00:58 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 21 12:00:58 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Jan 21 12:00:58 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Jan 21 12:00:58 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 21 12:00:58 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sat Jan 21 12:00:58 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 21 12:00:58 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Jan 21 16:00:45 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 21 16:00:45 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Jan 21 16:00:45 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 21 16:00:45 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Jan 21 16:00:45 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Jan 21 16:00:45 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 21 16:00:45 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sat Jan 21 16:00:45 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 21 16:00:45 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Jan 21 20:00:46 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 21 20:00:46 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Jan 21 20:00:46 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 21 20:00:46 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Jan 21 20:00:46 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Jan 21 20:00:46 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 21 20:00:46 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sat Jan 21 20:00:46 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 21 20:00:46 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Jan 22 08:02:37 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 22 08:02:37 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Jan 22 08:02:37 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 22 08:02:37 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Jan 22 08:02:37 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Jan 22 08:02:37 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 22 08:02:37 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sun Jan 22 08:02:37 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 22 08:02:37 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Jan 22 12:00:47 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 22 12:00:47 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Jan 22 12:00:47 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 22 12:00:47 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Jan 22 12:00:47 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Jan 22 12:00:47 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 22 12:00:47 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sun Jan 22 12:00:47 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 22 12:00:47 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Jan 22 16:01:11 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 22 16:01:11 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Jan 22 16:01:11 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 22 16:01:11 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Jan 22 16:01:11 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Jan 22 16:01:11 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 22 16:01:11 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sun Jan 22 16:01:11 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 22 16:01:11 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Jan 22 20:01:08 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 22 20:01:08 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Jan 22 20:01:08 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 22 20:01:08 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Jan 22 20:01:08 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Jan 22 20:01:08 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 22 20:01:08 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sun Jan 22 20:01:08 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 22 20:01:08 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Jan 23 08:02:29 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 23 08:02:29 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Jan 23 08:02:29 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 23 08:02:29 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Jan 23 08:02:29 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Jan 23 08:02:29 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 23 08:02:29 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Mon Jan 23 08:02:29 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 23 08:02:29 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Jan 23 12:00:48 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 23 12:00:48 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Jan 23 12:00:48 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 23 12:00:48 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Jan 23 12:00:48 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Jan 23 12:00:48 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 23 12:00:48 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Mon Jan 23 12:00:48 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 23 12:00:48 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Jan 23 16:01:35 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 23 16:01:36 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Jan 23 16:01:36 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 23 16:01:36 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Jan 23 16:01:36 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Jan 23 16:01:36 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 23 16:01:36 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Mon Jan 23 16:01:36 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 23 16:01:36 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Jan 23 20:00:47 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 23 20:00:47 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Jan 23 20:00:47 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 23 20:00:47 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Jan 23 20:00:47 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Jan 23 20:00:47 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 23 20:00:47 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Mon Jan 23 20:00:47 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 23 20:00:47 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Jan 24 08:03:01 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 24 08:03:01 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Jan 24 08:03:01 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 24 08:03:01 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Jan 24 08:03:01 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Jan 24 08:03:01 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 24 08:03:01 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Tue Jan 24 08:03:01 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 24 08:03:01 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Jan 24 12:02:36 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 24 12:02:36 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Jan 24 12:02:36 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 24 12:02:36 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Jan 24 12:02:36 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Jan 24 12:02:36 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 24 12:02:36 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Tue Jan 24 12:02:36 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 24 12:02:36 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Jan 24 16:01:14 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 24 16:01:14 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Jan 24 16:01:14 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 24 16:01:14 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Jan 24 16:01:14 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Jan 24 16:01:14 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 24 16:01:14 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Tue Jan 24 16:01:14 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 24 16:01:14 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Jan 24 20:01:47 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 24 20:01:47 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Jan 24 20:01:47 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 24 20:01:47 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Jan 24 20:01:47 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Jan 24 20:01:47 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 24 20:01:47 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Tue Jan 24 20:01:47 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 24 20:01:47 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Jan 25 08:01:55 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Jan 25 08:01:55 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Jan 25 08:01:55 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Jan 25 08:01:55 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Jan 25 08:01:55 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Jan 25 08:01:55 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Jan 25 08:01:55 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Wed Jan 25 08:01:55 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Jan 25 08:01:55 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Jan 25 12:00:56 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Jan 25 12:00:56 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Jan 25 12:00:56 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Jan 25 12:00:56 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Jan 25 12:00:56 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Jan 25 12:00:56 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Jan 25 12:00:56 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Wed Jan 25 12:00:56 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Jan 25 12:00:56 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Jan 25 16:00:53 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Jan 25 16:00:54 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Jan 25 16:00:54 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Jan 25 16:00:54 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Jan 25 16:00:54 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Jan 25 16:00:54 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Jan 25 16:00:54 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Wed Jan 25 16:00:54 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Jan 25 16:00:54 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Jan 25 20:00:49 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Jan 25 20:00:50 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Jan 25 20:00:50 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Jan 25 20:00:50 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Jan 25 20:00:50 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Jan 25 20:00:50 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Jan 25 20:00:50 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Wed Jan 25 20:00:50 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Jan 25 20:00:50 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Thu Jan 26 08:05:16 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Jan 26 08:05:17 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Thu Jan 26 08:05:17 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Jan 26 08:05:17 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Thu Jan 26 08:05:17 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Thu Jan 26 08:05:17 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Jan 26 08:05:17 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Thu Jan 26 08:05:17 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Jan 26 08:05:17 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Thu Jan 26 12:01:00 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Jan 26 12:01:00 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Thu Jan 26 12:01:00 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Jan 26 12:01:00 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Thu Jan 26 12:01:00 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Thu Jan 26 12:01:00 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Jan 26 12:01:00 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Thu Jan 26 12:01:00 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Jan 26 12:01:00 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Thu Jan 26 16:00:47 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Jan 26 16:00:47 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Thu Jan 26 16:00:47 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Jan 26 16:00:47 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Thu Jan 26 16:00:47 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Thu Jan 26 16:00:47 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Jan 26 16:00:47 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Thu Jan 26 16:00:47 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Jan 26 16:00:47 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Thu Jan 26 20:00:43 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Jan 26 20:00:43 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Thu Jan 26 20:00:43 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Jan 26 20:00:43 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Thu Jan 26 20:00:43 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Thu Jan 26 20:00:43 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Jan 26 20:00:43 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Thu Jan 26 20:00:43 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Jan 26 20:00:44 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Jan 27 08:06:13 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 27 08:06:13 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Jan 27 08:06:13 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 27 08:06:13 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Jan 27 08:06:13 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Jan 27 08:06:13 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 27 08:06:13 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Fri Jan 27 08:06:13 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 27 08:06:13 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Jan 27 12:02:03 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 27 12:02:03 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Jan 27 12:02:03 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 27 12:02:03 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Jan 27 12:02:03 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Jan 27 12:02:03 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 27 12:02:03 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Fri Jan 27 12:02:03 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 27 12:02:03 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Jan 27 16:00:59 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 27 16:01:00 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Jan 27 16:01:00 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 27 16:01:00 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Jan 27 16:01:00 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Jan 27 16:01:00 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 27 16:01:00 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Fri Jan 27 16:01:00 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 27 16:01:00 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Jan 27 20:00:50 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 27 20:00:50 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Jan 27 20:00:50 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 27 20:00:50 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Jan 27 20:00:51 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Jan 27 20:00:51 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 27 20:00:51 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Fri Jan 27 20:00:51 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Jan 27 20:00:51 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Jan 28 08:05:21 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 28 08:05:21 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Jan 28 08:05:21 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 28 08:05:21 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Jan 28 08:05:22 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Jan 28 08:05:22 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 28 08:05:22 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sat Jan 28 08:05:22 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 28 08:05:22 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Jan 28 12:01:49 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 28 12:01:49 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Jan 28 12:01:49 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 28 12:01:50 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Jan 28 12:01:50 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Jan 28 12:01:50 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 28 12:01:50 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sat Jan 28 12:01:50 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 28 12:01:50 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Jan 28 16:00:50 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 28 16:00:50 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Jan 28 16:00:50 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 28 16:00:50 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Jan 28 16:00:50 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Jan 28 16:00:50 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 28 16:00:50 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sat Jan 28 16:00:50 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 28 16:00:50 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Jan 28 20:00:50 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 28 20:00:50 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Jan 28 20:00:50 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 28 20:00:50 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Jan 28 20:00:50 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Jan 28 20:00:50 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 28 20:00:50 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sat Jan 28 20:00:50 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Jan 28 20:00:50 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Jan 29 08:07:03 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 29 08:07:04 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Jan 29 08:07:04 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 29 08:07:04 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Jan 29 08:07:04 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Jan 29 08:07:04 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 29 08:07:04 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sun Jan 29 08:07:04 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 29 08:07:04 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Jan 29 12:00:57 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 29 12:00:57 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Jan 29 12:00:57 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 29 12:00:57 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Jan 29 12:00:57 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Jan 29 12:00:57 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 29 12:00:57 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sun Jan 29 12:00:57 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 29 12:00:57 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Jan 29 16:00:48 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 29 16:00:48 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Jan 29 16:00:48 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 29 16:00:48 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Jan 29 16:00:48 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Jan 29 16:00:48 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 29 16:00:48 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sun Jan 29 16:00:48 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 29 16:00:48 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Jan 29 20:00:49 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 29 20:00:49 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Jan 29 20:00:49 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 29 20:00:49 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Jan 29 20:00:49 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Jan 29 20:00:49 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 29 20:00:49 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sun Jan 29 20:00:49 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Jan 29 20:00:49 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Jan 30 08:07:49 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 30 08:07:49 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Jan 30 08:07:49 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 30 08:07:49 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Jan 30 08:07:50 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Jan 30 08:07:50 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 30 08:07:50 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Mon Jan 30 08:07:50 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 30 08:07:50 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Jan 30 12:03:41 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 30 12:03:41 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Jan 30 12:03:41 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 30 12:03:41 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Jan 30 12:03:41 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Jan 30 12:03:41 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 30 12:03:41 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Mon Jan 30 12:03:41 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 30 12:03:41 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Jan 30 16:03:56 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 30 16:03:56 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Jan 30 16:03:56 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 30 16:03:56 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Jan 30 16:03:56 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Jan 30 16:03:56 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 30 16:03:56 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Mon Jan 30 16:03:56 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 30 16:03:56 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Jan 30 20:02:03 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 30 20:02:03 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Jan 30 20:02:03 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 30 20:02:03 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Jan 30 20:02:03 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Jan 30 20:02:03 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 30 20:02:03 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Mon Jan 30 20:02:03 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Jan 30 20:02:03 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Jan 31 08:07:30 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 31 08:07:31 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Jan 31 08:07:31 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 31 08:07:31 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Jan 31 08:07:32 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Jan 31 08:07:32 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 31 08:07:32 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Tue Jan 31 08:07:32 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 31 08:07:32 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Jan 31 12:04:02 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 31 12:04:02 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Jan 31 12:04:02 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 31 12:04:02 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Jan 31 12:04:02 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Jan 31 12:04:02 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 31 12:04:02 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Tue Jan 31 12:04:02 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 31 12:04:02 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Jan 31 16:02:07 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 31 16:02:07 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Jan 31 16:02:07 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 31 16:02:07 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Jan 31 16:02:07 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Jan 31 16:02:07 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 31 16:02:07 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Tue Jan 31 16:02:07 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 31 16:02:07 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Jan 31 20:01:01 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 31 20:01:01 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Jan 31 20:01:01 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 31 20:01:01 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Jan 31 20:01:01 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Jan 31 20:01:01 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 31 20:01:01 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Tue Jan 31 20:01:01 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Jan 31 20:01:01 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Feb 1 08:09:15 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 1 08:09:16 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Feb 1 08:09:16 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 1 08:09:16 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Feb 1 08:09:16 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Feb 1 08:09:16 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 1 08:09:16 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Wed Feb 1 08:09:16 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 1 08:09:16 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Feb 1 12:02:30 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 1 12:02:31 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Feb 1 12:02:31 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 1 12:02:31 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Feb 1 12:02:31 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Feb 1 12:02:31 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 1 12:02:31 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Wed Feb 1 12:02:31 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 1 12:02:31 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Feb 1 16:00:57 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 1 16:00:57 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Feb 1 16:00:57 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 1 16:00:57 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Feb 1 16:00:57 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Feb 1 16:00:57 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 1 16:00:57 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Wed Feb 1 16:00:57 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 1 16:00:57 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Feb 1 20:00:48 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 1 20:00:48 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Feb 1 20:00:48 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 1 20:00:48 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Feb 1 20:00:48 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Feb 1 20:00:48 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 1 20:00:48 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Wed Feb 1 20:00:48 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 1 20:00:48 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Thu Feb 2 08:11:37 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 2 08:11:37 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Thu Feb 2 08:11:37 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 2 08:11:37 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Thu Feb 2 08:11:37 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Thu Feb 2 08:11:37 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 2 08:11:38 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Thu Feb 2 08:11:38 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 2 08:11:39 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Thu Feb 2 12:02:37 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 2 12:02:38 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Thu Feb 2 12:02:38 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 2 12:02:38 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Thu Feb 2 12:02:38 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Thu Feb 2 12:02:38 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 2 12:02:38 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Thu Feb 2 12:02:38 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 2 12:02:38 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Thu Feb 2 16:01:05 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 2 16:01:05 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Thu Feb 2 16:01:05 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 2 16:01:05 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Thu Feb 2 16:01:05 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Thu Feb 2 16:01:05 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 2 16:01:05 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Thu Feb 2 16:01:05 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 2 16:01:05 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Thu Feb 2 20:00:54 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 2 20:00:54 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Thu Feb 2 20:00:54 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 2 20:00:54 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Thu Feb 2 20:00:54 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Thu Feb 2 20:00:54 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 2 20:00:54 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Thu Feb 2 20:00:54 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 2 20:00:54 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Feb 3 08:11:10 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Feb 3 08:11:11 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Feb 3 08:11:11 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Feb 3 08:11:11 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Feb 3 08:11:11 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Feb 3 08:11:11 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Feb 3 08:11:11 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Fri Feb 3 08:11:11 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Feb 3 08:11:12 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Feb 3 12:02:26 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Feb 3 12:02:26 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Feb 3 12:02:26 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Feb 3 12:02:26 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Feb 3 12:02:26 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Feb 3 12:02:26 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Feb 3 12:02:26 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Fri Feb 3 12:02:26 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Feb 3 12:02:26 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Feb 3 16:00:53 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Feb 3 16:00:53 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Feb 3 16:00:53 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Feb 3 16:00:53 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Feb 3 16:00:53 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Feb 3 16:00:53 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Feb 3 16:00:53 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Fri Feb 3 16:00:53 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Feb 3 16:00:53 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Feb 3 20:00:57 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Feb 3 20:00:57 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Feb 3 20:00:57 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Feb 3 20:00:57 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Feb 3 20:00:57 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Feb 3 20:00:57 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Feb 3 20:00:57 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Fri Feb 3 20:00:57 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Feb 3 20:00:57 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Feb 4 08:07:08 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Feb 4 08:07:08 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Feb 4 08:07:08 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Feb 4 08:07:08 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Feb 4 08:07:08 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Feb 4 08:07:09 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Feb 4 08:07:09 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sat Feb 4 08:07:09 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Feb 4 08:07:09 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Feb 4 12:02:27 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Feb 4 12:02:27 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Feb 4 12:02:27 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Feb 4 12:02:27 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Feb 4 12:02:27 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Feb 4 12:02:27 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Feb 4 12:02:27 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sat Feb 4 12:02:27 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Feb 4 12:02:27 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Feb 4 16:01:07 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Feb 4 16:01:07 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Feb 4 16:01:07 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Feb 4 16:01:07 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Feb 4 16:01:07 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Feb 4 16:01:07 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Feb 4 16:01:07 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sat Feb 4 16:01:07 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Feb 4 16:01:07 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Feb 4 20:00:59 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Feb 4 20:00:59 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Feb 4 20:00:59 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Feb 4 20:00:59 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Feb 4 20:00:59 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Feb 4 20:00:59 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Feb 4 20:00:59 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sat Feb 4 20:00:59 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Feb 4 20:00:59 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Feb 5 08:07:28 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Feb 5 08:07:28 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Feb 5 08:07:28 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Feb 5 08:07:29 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Feb 5 08:07:29 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Feb 5 08:07:29 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Feb 5 08:07:29 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sun Feb 5 08:07:29 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Feb 5 08:07:29 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Feb 5 12:02:30 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Feb 5 12:02:30 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Feb 5 12:02:30 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Feb 5 12:02:30 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Feb 5 12:02:30 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Feb 5 12:02:30 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Feb 5 12:02:30 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sun Feb 5 12:02:30 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Feb 5 12:02:30 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Feb 5 16:01:01 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Feb 5 16:01:01 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Feb 5 16:01:01 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Feb 5 16:01:01 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Feb 5 16:01:01 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Feb 5 16:01:01 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Feb 5 16:01:01 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sun Feb 5 16:01:01 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Feb 5 16:01:01 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Feb 5 20:00:55 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Feb 5 20:00:56 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Feb 5 20:00:56 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Feb 5 20:00:56 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Feb 5 20:00:56 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Feb 5 20:00:56 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Feb 5 20:00:56 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sun Feb 5 20:00:56 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Feb 5 20:00:56 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Feb 6 08:12:58 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Feb 6 08:13:00 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Feb 6 08:13:00 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Feb 6 08:13:00 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Feb 6 08:13:00 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Feb 6 08:13:00 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Feb 6 08:13:00 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Mon Feb 6 08:13:00 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Feb 6 08:13:00 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Feb 6 12:02:26 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Feb 6 12:02:26 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Feb 6 12:02:26 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Feb 6 12:02:26 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Feb 6 12:02:26 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Feb 6 12:02:26 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Feb 6 12:02:26 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Mon Feb 6 12:02:26 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Feb 6 12:02:26 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Feb 6 16:01:01 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Feb 6 16:01:01 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Feb 6 16:01:01 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Feb 6 16:01:01 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Feb 6 16:01:01 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Feb 6 16:01:01 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Feb 6 16:01:01 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Mon Feb 6 16:01:01 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Feb 6 16:01:01 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Feb 6 20:00:55 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Feb 6 20:00:55 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Feb 6 20:00:55 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Feb 6 20:00:55 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Feb 6 20:00:55 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Feb 6 20:00:55 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Feb 6 20:00:55 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Mon Feb 6 20:00:55 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Feb 6 20:00:55 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Feb 7 08:20:10 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Feb 7 08:20:11 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Feb 7 08:20:12 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Feb 7 08:20:12 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Feb 7 08:20:12 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Feb 7 08:20:14 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Feb 7 08:20:15 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Tue Feb 7 08:20:15 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Feb 7 08:20:16 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Feb 7 12:03:04 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Feb 7 12:03:04 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Feb 7 12:03:04 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Feb 7 12:03:04 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Feb 7 12:03:04 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Feb 7 12:03:04 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Feb 7 12:03:04 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Tue Feb 7 12:03:04 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Feb 7 12:03:04 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Feb 7 16:01:04 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Feb 7 16:01:04 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Feb 7 16:01:04 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Feb 7 16:01:04 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Feb 7 16:01:04 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Feb 7 16:01:04 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Feb 7 16:01:04 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Tue Feb 7 16:01:05 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Feb 7 16:01:05 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Feb 7 20:01:00 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Feb 7 20:01:00 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Feb 7 20:01:00 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Feb 7 20:01:00 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Feb 7 20:01:00 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Feb 7 20:01:00 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Feb 7 20:01:00 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Tue Feb 7 20:01:00 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Feb 7 20:01:00 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Feb 8 08:20:35 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 8 08:20:35 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Feb 8 08:20:35 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 8 08:20:37 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Feb 8 08:20:38 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Feb 8 08:20:38 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 8 08:20:38 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Wed Feb 8 08:20:38 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 8 08:20:38 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Feb 8 12:02:57 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 8 12:02:57 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Feb 8 12:02:57 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 8 12:02:58 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Feb 8 12:02:58 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Feb 8 12:02:58 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 8 12:02:58 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Wed Feb 8 12:02:58 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 8 12:02:58 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Feb 8 16:01:08 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 8 16:01:08 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Feb 8 16:01:08 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 8 16:01:08 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Feb 8 16:01:08 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Feb 8 16:01:08 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 8 16:01:08 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Wed Feb 8 16:01:08 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 8 16:01:08 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Feb 8 20:00:56 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 8 20:00:56 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Feb 8 20:00:56 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 8 20:00:56 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Feb 8 20:00:56 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Feb 8 20:00:56 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 8 20:00:56 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Wed Feb 8 20:00:56 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 8 20:00:56 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Thu Feb 9 08:08:22 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 9 08:08:23 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Thu Feb 9 08:08:23 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 9 08:08:23 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Thu Feb 9 08:08:23 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Thu Feb 9 08:08:23 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 9 08:08:23 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Thu Feb 9 08:08:24 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 9 08:08:24 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Thu Feb 9 12:02:43 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 9 12:02:44 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Thu Feb 9 12:02:44 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 9 12:02:44 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Thu Feb 9 12:02:44 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Thu Feb 9 12:02:44 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 9 12:02:44 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Thu Feb 9 12:02:44 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 9 12:02:44 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Thu Feb 9 16:01:10 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 9 16:01:10 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Thu Feb 9 16:01:10 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 9 16:01:10 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Thu Feb 9 16:01:10 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Thu Feb 9 16:01:10 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 9 16:01:10 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Thu Feb 9 16:01:10 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 9 16:01:10 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Thu Feb 9 20:01:01 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 9 20:01:02 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Thu Feb 9 20:01:02 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 9 20:01:02 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Thu Feb 9 20:01:02 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Thu Feb 9 20:01:02 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 9 20:01:02 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Thu Feb 9 20:01:03 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 9 20:01:03 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Feb 10 08:17:23 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Feb 10 08:17:23 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Feb 10 08:17:23 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Feb 10 08:17:25 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Feb 10 08:17:25 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Feb 10 08:17:25 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Feb 10 08:17:25 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Fri Feb 10 08:17:25 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Feb 10 08:17:25 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Feb 10 12:03:20 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Feb 10 12:03:21 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Feb 10 12:03:21 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Feb 10 12:03:21 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Feb 10 12:03:21 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Feb 10 12:03:21 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Feb 10 12:03:21 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Fri Feb 10 12:03:21 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Feb 10 12:03:21 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Feb 10 16:01:02 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Feb 10 16:01:02 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Feb 10 16:01:02 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Feb 10 16:01:02 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Feb 10 16:01:02 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Feb 10 16:01:02 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Feb 10 16:01:03 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Fri Feb 10 16:01:03 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Feb 10 16:01:03 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Feb 10 20:01:05 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Feb 10 20:01:05 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Feb 10 20:01:05 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Feb 10 20:01:06 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Feb 10 20:01:06 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Feb 10 20:01:06 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Feb 10 20:01:06 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Fri Feb 10 20:01:06 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Feb 10 20:01:06 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Feb 11 08:12:15 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Feb 11 08:12:16 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Feb 11 08:12:16 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Feb 11 08:12:16 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Feb 11 08:12:16 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Feb 11 08:12:16 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Feb 11 08:12:16 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sat Feb 11 08:12:16 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Feb 11 08:12:16 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Feb 11 12:02:25 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Feb 11 12:02:25 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Feb 11 12:02:25 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Feb 11 12:02:25 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Feb 11 12:02:25 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Feb 11 12:02:25 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Feb 11 12:02:25 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sat Feb 11 12:02:25 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Feb 11 12:02:25 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Feb 11 16:00:58 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Feb 11 16:00:58 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Feb 11 16:00:58 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Feb 11 16:00:58 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Feb 11 16:00:58 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Feb 11 16:00:58 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Feb 11 16:00:58 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sat Feb 11 16:00:58 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Feb 11 16:00:58 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Feb 11 20:00:53 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Feb 11 20:00:53 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sat Feb 11 20:00:53 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Feb 11 20:00:53 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Feb 11 20:00:53 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sat Feb 11 20:00:53 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Feb 11 20:00:53 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sat Feb 11 20:00:53 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sat Feb 11 20:00:53 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Feb 12 08:08:40 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Feb 12 08:08:40 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Feb 12 08:08:40 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Feb 12 08:08:40 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Feb 12 08:08:40 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Feb 12 08:08:40 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Feb 12 08:08:40 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sun Feb 12 08:08:40 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Feb 12 08:08:40 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Feb 12 12:02:32 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Feb 12 12:02:32 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Feb 12 12:02:32 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Feb 12 12:02:32 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Feb 12 12:02:32 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Feb 12 12:02:32 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Feb 12 12:02:32 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sun Feb 12 12:02:32 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Feb 12 12:02:32 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Feb 12 16:01:02 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Feb 12 16:01:02 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Feb 12 16:01:02 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Feb 12 16:01:02 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Feb 12 16:01:02 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Feb 12 16:01:02 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Feb 12 16:01:02 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sun Feb 12 16:01:02 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Feb 12 16:01:02 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Feb 12 20:00:57 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Feb 12 20:00:57 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Sun Feb 12 20:00:57 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Feb 12 20:00:57 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Feb 12 20:00:57 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Sun Feb 12 20:00:57 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Feb 12 20:00:57 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Sun Feb 12 20:00:57 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Sun Feb 12 20:00:57 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Feb 13 08:16:01 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Feb 13 08:16:01 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Feb 13 08:16:02 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Feb 13 08:16:02 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Feb 13 08:16:02 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Feb 13 08:16:02 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Feb 13 08:16:02 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Mon Feb 13 08:16:02 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Feb 13 08:16:02 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Feb 13 12:02:45 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Feb 13 12:02:45 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Feb 13 12:02:45 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Feb 13 12:02:45 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Feb 13 12:02:45 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Feb 13 12:02:45 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Feb 13 12:02:45 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Mon Feb 13 12:02:45 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Feb 13 12:02:45 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Feb 13 16:01:15 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Feb 13 16:01:15 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Feb 13 16:01:15 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Feb 13 16:01:15 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Feb 13 16:01:15 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Feb 13 16:01:15 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Feb 13 16:01:15 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Mon Feb 13 16:01:15 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Feb 13 16:01:15 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Feb 13 20:00:56 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Feb 13 20:00:57 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Mon Feb 13 20:00:57 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Feb 13 20:00:57 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Feb 13 20:00:57 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Mon Feb 13 20:00:57 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Feb 13 20:00:57 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Mon Feb 13 20:00:57 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Mon Feb 13 20:00:57 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Feb 14 08:17:57 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Feb 14 08:17:57 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Feb 14 08:17:57 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Feb 14 08:17:57 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Feb 14 08:17:57 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Feb 14 08:17:57 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Feb 14 08:17:57 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Tue Feb 14 08:17:57 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Feb 14 08:17:57 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Feb 14 12:03:06 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Feb 14 12:03:06 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Feb 14 12:03:06 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Feb 14 12:03:06 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Feb 14 12:03:06 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Feb 14 12:03:06 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Feb 14 12:03:06 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Tue Feb 14 12:03:06 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Feb 14 12:03:06 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Feb 14 16:01:20 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Feb 14 16:01:20 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Feb 14 16:01:20 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Feb 14 16:01:20 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Feb 14 16:01:20 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Feb 14 16:01:20 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Feb 14 16:01:20 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Tue Feb 14 16:01:20 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Feb 14 16:01:20 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Feb 14 20:01:02 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Feb 14 20:01:02 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Tue Feb 14 20:01:02 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Feb 14 20:01:02 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Feb 14 20:01:02 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Tue Feb 14 20:01:02 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Feb 14 20:01:02 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Tue Feb 14 20:01:02 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Tue Feb 14 20:01:02 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Feb 15 08:24:28 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 15 08:24:29 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Feb 15 08:24:30 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 15 08:24:30 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Feb 15 08:24:30 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Feb 15 08:24:30 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 15 08:24:30 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Wed Feb 15 08:24:32 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 15 08:24:32 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Feb 15 12:04:56 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 15 12:04:56 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Feb 15 12:04:56 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 15 12:04:56 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Feb 15 12:04:56 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Feb 15 12:04:56 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 15 12:04:56 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Wed Feb 15 12:04:56 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 15 12:04:56 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Feb 15 16:01:26 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 15 16:01:26 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Feb 15 16:01:26 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 15 16:01:26 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Feb 15 16:01:26 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Feb 15 16:01:26 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 15 16:01:26 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Wed Feb 15 16:01:26 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 15 16:01:26 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Feb 15 19:27:58 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 15 19:27:58 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Feb 15 19:27:58 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 15 19:27:58 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Feb 15 19:27:58 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Feb 15 19:27:58 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 15 19:27:58 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Wed Feb 15 19:27:58 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 15 19:27:58 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Feb 15 20:12:31 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 15 20:12:31 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Wed Feb 15 20:12:32 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 15 20:12:32 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Feb 15 20:12:32 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Wed Feb 15 20:12:32 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 15 20:12:32 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Wed Feb 15 20:12:32 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Wed Feb 15 20:12:32 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Thu Feb 16 08:43:25 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 16 08:43:27 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Thu Feb 16 08:43:28 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 16 08:43:28 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Thu Feb 16 08:43:28 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Thu Feb 16 08:43:28 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 16 08:43:28 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Thu Feb 16 08:43:28 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Thu Feb 16 08:43:29 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 11:16:02 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Mar 31 16:33:19 2006 Subject: [egenix-users] Calc size of Python objects Message-ID: <3DD0D4E2.8070603@spirito.de> hi, in mxTools there is a nice function called sizeof() to calculate the amount of memory an object uses, but that's only true for simple objects like strings and integers I believe. but I'm interested in the total amount a complex object uses. is this the right way to do it? def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) elif type(i) == type([]): for v in i: s += calcsize(v) return s BTW, does someone have an idea how to get aware of the total memory usage of a running python program within itself? the module "resource" doesn't work on my machine. thanks, dirk From mal at lemburg.com Tue Nov 12 11:33:50 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Mar 31 16:33:19 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> Message-ID: <3DD0D90E.3090400@lemburg.com> Dirk Holtwick wrote: > hi, > > in mxTools there is a nice function called sizeof() to calculate the > amount of memory an object uses, but that's only true for simple objects > like strings and integers I believe. True. sizeof() only measures the size of the bare PyObject, not including any possibly referenced memory. Since dictionaries, classes, instances etc. use extra memory for keeping data, the value returned by sizeof() is not correct. Deep knowledge of how Python does its memory allocation is needed to figure this out. > but I'm interested in the total > amount a complex object uses. is this the right way to do it? > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) + you have to add the memory for the dictionary table itself; that's 2 PyObject pointers per slot. The table size depends on the size of the dictionary. > elif type(i) == type([]): > for v in i: > s += calcsize(v) For lists, sizeof() does not include the table of entries, so you'll have to add one PyObject pointer per entry. For tuples, sizeof() does include the table size. > return s > > BTW, does someone have an idea how to get aware of the total memory > usage of a running python program within itself? the module "resource" > doesn't work on my machine. Take a look at mx/ODBC/Misc/proc.py. That module is not open-source, though, as it's part of mxODBC. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From holtwick at spirito.de Tue Nov 12 12:12:31 2002 From: holtwick at spirito.de (Dirk Holtwick) Date: Fri Mar 31 16:33:19 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> Message-ID: <3DD0E21F.9050406@spirito.de> So this one would be more accurate? If I understand you right, it's not possible to calculate the exact amount of memory used by a python object? class myPyObject: pass def calcsize(i, s=0): s = sizeof(i) if type(i) == type({}): for k, v in i.items(): s += calcsize(k) s += calcsize(v) s += 2 * sizeof(myPyObject()) elif type(i) == type([]): for v in i: s += calcsize(v) s += 1 * sizeof(myPyObject()) elif type(i) == type((1,)): for v in i: s += calcsize(v) return s I don't know exctly how to determine the size of a PyObject. The proc.py is exactly what I was looking for and it's working fine. Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs in my application. yours, dirk From mal at lemburg.com Tue Nov 12 12:29:58 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Mar 31 16:33:19 2006 Subject: [egenix-users] Calc size of Python objects References: <3DD0D4E2.8070603@spirito.de> <3DD0D90E.3090400@lemburg.com> <3DD0E21F.9050406@spirito.de> Message-ID: <3DD0E636.7040305@lemburg.com> Dirk Holtwick wrote: > So this one would be more accurate? If I understand you right, it's not > possible to calculate the exact amount of memory used by a python object? > > class myPyObject: > pass > > def calcsize(i, s=0): > s = sizeof(i) > if type(i) == type({}): > for k, v in i.items(): > s += calcsize(k) > s += calcsize(v) > s += 2 * sizeof(myPyObject()) Well, the table size will usually be around 3/2 of the number of entries. For more details see the C implementation :-) > elif type(i) == type([]): > for v in i: > s += calcsize(v) > s += 1 * sizeof(myPyObject()) > elif type(i) == type((1,)): > for v in i: > s += calcsize(v) > return s > > I don't know exctly how to determine the size of a PyObject. import struct sizeof_PyObject = len(struct.pack('P', 0)) On 32-bit machines that's usually 4 bytes. > The proc.py is exactly what I was looking for and it's working fine. > Thanks a lot Marc-Andre! Helps me tracking down some strange memory bugs > in my application. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From donna at v1.wustl.edu Mon Nov 18 13:01:47 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Mar 31 16:33:19 2006 Subject: [egenix-users] mx.DateTime:Solaris:gcc3.2:problem on import Message-ID: <3DD9391B.FABE2E3C@v1.wustl.edu> Hi, When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup seems to go fine. It builds and installs smoothly. But when I import mx.DateTime, python crashes: Python 2.1.3 (#1, Nov 18 2002, 11:16:06) [GCC 3.2] on sunos5 Type "copyright", "credits" or "license" for more information. >>> import mx.DateTime *** You don't have the (right) mxDateTime binaries installed ! Traceback (most recent call last): File "", line 1, in ? File "mx/DateTime/__init__.py", line 8, in ? from DateTime import * File "mx/DateTime/DateTime.py", line 9, in ? from mxDateTime import * File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? raise ImportError, why ImportError: No module named mxDateTime >>> But if I rebuild egenix-mx-base-2.0.3.tar.gz using exactly the same steps, except using gcc 2.95 instead of 3.2, it works fine -- no problems at all. Since I have a working version, this is not at all urgent, as far as I am concerned. I would be happy to try troubleshooting the problem, but I'm not handy at all with either gdb or pdb. Thanks, Donna From donna at v1.wustl.edu Tue Nov 19 08:12:10 2002 From: donna at v1.wustl.edu (Donna Hanlon) Date: Fri Mar 31 16:33:19 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import References: <200211191101.gAJB10008478@www.egenix.com> Message-ID: <3DDA46BA.AEE1F2ED@v1.wustl.edu> Actually, this is what happens when I try this in the build directory (which just happens to have an mx/DateTime subdirectory). ;-) When I'm smart enough to cd out of the build directory, I get "Bus error (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine with the gcc2.95-built module. On Mon, 18 Nov 2002, Donna Hanlon wrote: > When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > seems to go fine. It builds and installs smoothly. But when I import > mx.DateTime, python crashes: > > Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > [GCC 3.2] on sunos5 > Type "copyright", "credits" or "license" for more information. > >>> import mx.DateTime > *** You don't have the (right) mxDateTime binaries installed ! > Traceback (most recent call last): > File "", line 1, in ? > File "mx/DateTime/__init__.py", line 8, in ? > from DateTime import * > File "mx/DateTime/DateTime.py", line 9, in ? > from mxDateTime import * > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > raise ImportError, why > ImportError: No module named mxDateTime From mal at lemburg.com Tue Nov 19 15:20:47 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Mar 31 16:33:19 2006 Subject: [egenix-users] Re: mx.DateTime:Solaris:gcc3.2:problem on import In-Reply-To: <200211191101.gAJB10008478@www.egenix.com> References: <200211191101.gAJB10008478@www.egenix.com> <3DDA46BA.AEE1F2ED@v1.wustl.edu> Message-ID: <3DDA48BF.3090104@lemburg.com> Donna Hanlon wrote: > Actually, this is what happens when I try this in the build directory > (which just happens to have an mx/DateTime subdirectory). ;-) > > When I'm smart enough to cd out of the build directory, I get "Bus error > (core dumped)" when I import the gcc3.2-built mx.DateTime; works fine > with the gcc2.95-built module. You should always CD out of the build dir to test the packages, since otherwise, Python picks up the mx dir in the build dir (which doesn't include the compiled extensions). The bus error sounds like a problem, though. Have you built Python using GCC 3.2 or 2.95 ? > > On Mon, 18 Nov 2002, Donna Hanlon wrote: > > >When I build egenix-mx-base-2.0.3.tar.gz on SunOS 5.8 Generic_108528-14 > >sun4u sparc SUNW,Ultra-5_10 using gcc 3.2, using python 2.1.3, setup > >seems to go fine. It builds and installs smoothly. But when I import > >mx.DateTime, python crashes: > > > >Python 2.1.3 (#1, Nov 18 2002, 11:16:06) > >[GCC 3.2] on sunos5 > >Type "copyright", "credits" or "license" for more information. > > > >>>>import mx.DateTime > > > >*** You don't have the (right) mxDateTime binaries installed ! > >Traceback (most recent call last): > > File "", line 1, in ? > > File "mx/DateTime/__init__.py", line 8, in ? > > from DateTime import * > > File "mx/DateTime/DateTime.py", line 9, in ? > > from mxDateTime import * > > File "mx/DateTime/mxDateTime/__init__.py", line 13, in ? > > raise ImportError, why > >ImportError: No module named mxDateTime > > > _______________________________________________________________________ > eGenix.com User Mailing List http://www.egenix.com/ > http://lists.egenix.com/mailman/listinfo/egenix-users -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/ From malcolm at collegenet.com Tue Nov 19 10:02:39 2002 From: malcolm at collegenet.com (Malcolm Heath) Date: Fri Mar 31 16:33:19 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 Message-ID: <1037728959.4978.38.camel@racecar.unival.com> Hello all, I'm attempting to install the RPM egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The server is a RedHat 7.2 machine, fully updated, with python2.2 installed via RPM. The command "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" fails with the error: error: failed dependencies: libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 So, obviously, I'm missing some libraries. I then found and installed unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which satisfied the first dependency. Some further exploration of the egenix site told me that I also needed libraries from iodbc.org. I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. I'm not wondering how to fix this, particularly. What I am wondering, as a prospective customer of this software, is why the RPM requires both unixODBC and iodbc libraries? Are they actually required? We run a large shop here, and managing custom software is a big, and often frustrating part of that job. Having to download, compile or install, and keep track of yet another package that my developers don't actually need to use is wasteful of my time, and unnecessary. So, what I'd like to know is if simply doing a --force is going to get me over this hurdle, without leaving me with a non-function, or unstable install. Sincerely, Malcolm Heath From mal at lemburg.com Tue Nov 19 19:52:04 2002 From: mal at lemburg.com (M.-A. Lemburg) Date: Fri Mar 31 16:33:19 2006 Subject: [egenix-users] Install conflict, egenixs-mx-commercial-2.0.5 RPM on RH 7.2 In-Reply-To: <1037728959.4978.38.camel@racecar.unival.com> References: <1037728959.4978.38.camel@racecar.unival.com> Message-ID: <3DDA8854.6040302@lemburg.com> Malcolm Heath wrote: > Hello all, > > I'm attempting to install the RPM > egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm on a server here. The > server is a RedHat 7.2 machine, fully updated, with python2.2 installed > via RPM. The command > > "rpm -ivh egenix-mx-commercial-2.0.5-py2.2_1.i386.rpm" > > fails with the error: > > error: failed dependencies: > libiodbc.so.2 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > libodbc.so.1 is needed by egenix-mx-commercial-2.0.5-py2.2_1 > > So, obviously, I'm missing some libraries. I then found and installed > unixODBC-devel-2.0.7-3.i386.rpm and unixODBC-2.0.7-3.i386.rpm, which > satisfied the first dependency. Some further exploration of the egenix > site told me that I also needed libraries from iodbc.org. > > I downloaded, and attempted to install libiodbc-3.0.6-2.i386-glibc21.rpm > but that failed, conflicting with files from unixODBC-2.0.7-3.i386.rpm. > > I'm not wondering how to fix this, particularly. What I am wondering, as > a prospective customer of this software, is why the RPM requires both > unixODBC and iodbc libraries? Are they actually required? No, only one of them is required. The RPM contains subpackages for both managers since one of them is usually installed on the target system, so rpm -i --nodeps egenix-mx-commercial-*.rpm will work regardless of which one you have installed. We ship the RPM bundled for two ODBC managers to reduce the confusion on the user side. The instructions on the download page mentions this: http://www.lemburg.com/files/python/eGenix-mx-Extensions.html#Install-mxCOMMERCIAL > We run a large shop here, and managing custom software is a big, and > often frustrating part of that job. Having to download, compile or > install, and keep track of yet another package that my developers don't > actually need to use is wasteful of my time, and unnecessary. > > So, what I'd like to know is if simply doing a --force is going to get > me over this hurdle, without leaving me with a non-function, or unstable > install. Doing so is OK. Just tell your developers which subpackage of mxODBC to program against (unixODBC in your case). BTW, the most current unixODBC version is 2.2.3. That version fixes quite a few bugs in that ODBC manager. I usually tell people to start with iODBC 3.0.6 and only switch to unixODBC in case some driver doesn't like iODBC. iODBC has proven to be much more stable and standards conform than unixODBC. unixODBC is only necessary in case you plan to do Unicode over ODBC -- iODBC currently does not support Unicode. -- Marc-Andre Lemburg CEO eGenix.com Software GmbH _______________________________________________________________________ eGenix.com -- Makers of the Python mx Extensions: mxDateTime,mxODBC,... Python Consulting: http://www.egenix.com/ Python Software: http://www.egenix.com/files/python/